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Abstract 

Despite the pressure from potential employers and higher education administrators to develop 
students’ global and intercultural competence, traditional study abroad programs simply are not 
feasible for many postsecondary students (Berdan & Johannes, 2014; Fischer, 2015). The purpose 
of this study was to evaluate the effectiveness of an online delivery model for study abroad 
activities. Building upon the findings of an initial exploratory program using Adobe Connect web 
conferencing tools, this evaluative case study was the second in a series of design based research 
studies intended to identify effective practices and develop recommendations to further refine the 
model through an iterative evaluation process. Using the Online Learning Consortium’s Quality 
Framework, each of the Five Pillars that support successful online learning (access, student 
satisfaction, faculty satisfaction, learning effectiveness, & scale) was evaluated through a 
combination of anonymous surveys, pre/post assessments, observations, and student & instructor 
interviews (Moore, 2005). 

Regarding access, 26 students who were enrolled in an intercultural communication course 
were able to participate in a study abroad experience in Italy; 10 students participated in the 
traditional study abroad trip in Italy while the other 16 participated virtually. The online students 
were able to join the live meetings, thus expanding their access to international experiences that 
normally would be closed to them. In tenns of student and faculty satisfaction, both groups of 
students and the instructor reported specific areas of satisfaction, offered critical feedback, and felt 
that the concept was a viable one. While the students who traveled to Italy had a far more 
immersive experience, both groups demonstrated gains in learning. Using Morais and Ogden’s 
(2010) global citizenship pre/post assessment, both groups showed improvement on the self- 
awareness and intercultural communication scales, and when comparing the two groups the online 
students improved more on the social responsibility scale while the students who traveled 
improved more on the global knowledge scale. Both groups submitted assignments of similar 
quality, engaged in communications between the abroad and online groups, and interacted with 
the instructor and experts in the field. In tenns of differences in student engagement, students had 
differing opinions on the interaction with the technology and the online group asked more 
questions during live meetings. The implications of this pilot study should inform the planning of 
the next case evaluation and are important for other educators who wish to implement a similar 
approach to internationalizing the curriculum through online instruction. 

Keywords: Internationalization, study abroad, international education, online learning, distance 
education 
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Introduction 

In response to increasing demands by employers for globally prepared graduates, many 
institutions of higher education set strategic goals aimed at enhancing internationalization. Both 
employers and educators promote the importance of global and intercultural competence to success 
in our modem-day workforce, which has resulted in a push to increase enrollment in study abroad 
programs (Berdan & Johannes, 2014; Fischer, 2015). There are many obstacles to study abroad 
such as cost, lack of suitable opportunities, rigid degree pathways, and competing obligations from 
internships, work, or family. According to the Institute of International Education (2017) the 
national average of U.S. undergraduate students who traveled abroad during their degree program 
in 2016 was 10%. The student population that does study abroad typically does not reflect the 
diversity of most U.S. higher education institutions overall (Institute of International Education, 
2017). 

In tandem with efforts to internationalize the curriculum, online education has steadily 
increased as a strategic priority for higher education in the United States. According to the Babson 
Survey Research Group’s Grade Level: Tracking Online Education in the United States, over 70% 
of Chief Academic Officers that responded agreed that online education is critical for their 
institution’s long-tenn strategy (Allen & Seaman, 2016). Within a culture of growing technology 
acceptance, it would be logical to explore technology-based online programs that would directly 
benefit those students that simply cannot study abroad by offering access to high-impact 
international experiences without the barriers discussed above. 

The purpose of this study was to evaluate the effectiveness of using web conferencing tools 
to allow students to actively participate in live study abroad activities from home or on campus. 
Building upon the findings of Howard and Gunter's (2017) initial exploratory program using 
Adobe Connect, this evaluative case study is intended to further refine a technology-based model 
that expands student access to high-impact international experiences by connecting students in the 
U.S. with their peers, instructors, and experts abroad. 


Literature Review 

Traditional study abroad programs, including short-term programs, have an established 
history of demonstrated benefits for students (Chieffo & Griffiths, 2004; McKeown, 2009; Spencer 
& Tuma, 2002; Tarrant, Rubin, & Stoner, 2014; Vandeveer & Menefee, 2006). For example, 
Chieffo and Griffiths (2004) found that short-term study abroad programs had a positive impact 
on students’ intercultural awareness and functional knowledge. Tarrant, Rubin, & Stoner (2014) 
also found through experimental research that study abroad paired with academic focus yielded 
increases in global citizenship in a 4-week course. McKeown (2009) even coined the phrase “first 
time effect” to describe the profound transformational impact even one study abroad experience 
(regardless of length) may have on a student’s intellectual development. 

Internationalization efforts at home have also shown that students may develop 
intercultural competencies without the burden of traveling to a foreign country (Baldassar & 
McKenzie, 2016; Leask, 2004; Lilley, Barker, & Harris, 2015; Nilsson, 2003; Soria & Troisi, 
2013). Jones (2013) argues that all students, not just those with international mobility, should enjoy 
an internationalized curriculum in order to develop transferable skills that will allow them to 
compete in the current job market. Lilley, Barker, and Harris (2015) suggest that “...learning to 
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become a global citizen is a process that occurs in response to particular facilitating situations that 
could be simulated through mobility comparable learning experiences ‘at home’” (p. 242). 

In addition to trends in increased internationalization efforts in higher education, online 
learning is also on the rise. Allen and Seaman (2016) reported a multi-year trend in increased 
online enrollments in U.S. higher education while overall enrollment decreased. They also reported 
that more than one in four students (28%) take at least one online course and that over 60% of 
chief academic officers agreed that online education is critical for their institution’s long-tenn 
strategy (Allen & Seaman, 2016). 

In this culture of technology acceptance and upward trend in online learning, educators 
have turned to technology to expand internationalization of curriculum in innovative ways. Roberts 
and Monroe-Baillargeon (2012) describe how technology such as web-based videoconferencing, 
learning management systems, and social media sites allow for new multicultural online learning 
environments. Specifically, Roberts and Monroe-Baillargeon (2012) state, “The benefits of 
multiculturalism and cross-cultural exchange, once only available to individuals with the resources 
to travel abroad, are increasingly available through information technology and creative pedagogy. 
Those professors who effectively integrate technology in their teaching will now have the ability 
to reach across borders to create virtual multicultural learning communities” (p. 41). Scovotti and 
Spiller (2011) utilized synchronous and asynchronous technology to provide MBA students in the 
U.S. and Germany the opportunity to collaborate on a real-world business challenge at a distance, 
and they found that introducing video conferencing enhanced productivity and student satisfaction. 

Kenny and Lenz (2009) reported on a satellite-based project that allowed students in the 
classroom on the main campus to interact firsthand with their peers in the desert through a live 
broadcast from the field. One of the more interesting findings was that students reported increased 
attention levels when they were permitted to ask questions during the live broadcast (Kenny and 
Lenz, 2009). In a follow-up study that employed the same technology to broadcast from India, 
Kenny and Gunter (2015) found that the additional use of the text chat feature during the live 
broadcast appeared to increase student engagement and their sense of social presence. Howard and 
Gunter (2017) then initiated a series of formative case evaluations to develop a more cost-effective 
delivery mechanism for providing similar live broadcasts from the field with interactive chat. 
Advances in mobile technology now allow educators to take instruction outside of the classroom 
to facilitate learning in creative delivery methods with sound instructional strategies (Sung, Chang, 
& Liu, 2016). As a result, Howard and Gunter (2017) found that a valuable international learning 
experience could be provided to students online in the U.S. by connecting them with their peers 
on a traditional study abroad trip through web conferencing tools like Adobe Connect. 

The purpose of this study is to continue the cyclical, design-based research started by 
Howard and Gunter (2017) to refine and further develop the web conferencing based delivery of 
internationally-focused instruction through a series of case evaluations. Three guiding questions 
drove this formative case evaluation: (1) Did the program accomplish its goals? (2) What was the 
impact of how it was implemented? (3) What changes to the design are necessary to firmly 
establish it as a valuable alternative to traditional study abroad? These guiding questions generated 
a set of evaluation questions that were then organized using the Five Pillars of the Online Learning 
Consortium (OLC) Quality Framework (Moore, 2005): 
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Access: 

1. What is the potential impact of this program for providing students access to 
international learning opportunities? 

Student Satisfaction: 

2. What is the difference in student satisfaction with the overall experience between the 
online and abroad groups? 

3. What were online participants’ reactions to the web conferencing system? 

4. What was the impact of technology on the students’ ability to participate in remote 
instructional activities? 

Faculty Satisfaction: 

5. What were the instructor’s perceptions about his teaching experience with this 
technology-mediated approach? 

Learning Effectiveness: 

6. What is the difference in student engagement with activities and experts in the field 
between students who travel abroad (abroad group) and those who participate via Web 
conferencing (online group)? 

7. How were the learning experiences of the online and abroad groups similar and how 
were they different? 

Scale: 

8. How does the cost of this case compare to previous technology-mediated attempts? 

9. How can this approach be improved? 

Methods 

The Online Learning Consortium’s Quality Framework was used and each of the Five 
Pillars that support successful online learning was evaluated through a combination of quantitative 
and qualitative methods including anonymous surveys, observations, student and instructor 
interviews and Morais and Ogden’s (2010) global citizenship pre/post assessment (Moore, 2005). 
This fonnative case evaluation is the second in a series of iterative studies where the 
recommendations of the previous study are implemented and then evaluated. Based on Howard 
and Gunter’s (2017) study with a global health management course in Brazil, the methodology 
was replicated in this study with the addition of a pre/post global citizenship assessment to address 
potential learning outcomes in a more formal class structure. 

The instructional model that was evaluated mirrored Howard and Gunter’s (2017), which 
employed the Adobe Connect web conferencing tool to provide online participants in the U.S. 
access to the live study abroad activities in Italy, including tours of historical sites, cultural venues, 
and communities led by local experts. The online facilitator used an iPad with Adobe Connect to 
link the abroad group with the online group which enabled discussions between the groups using 
the chat, audio, and video functions. 

Once logged into the web conference, the online participants were able to view the video feed 
from the perspective of the online facilitator as though they were part of the group of students in 
the field. They could hear the live discussion, but their individual microphones remained muted 
until they were ready to speak to the group or type in the chat. All live sessions were recorded 
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and videos were captured and uploaded to the online course hosted in the university’s learning 
management system to allow for convenient access between live sessions. 

Ultimately, the instructor was responsible for delivering instruction for both the abroad and 
online groups. The online facilitator was responsible for all technical aspects of running the web 
conferencing tool and keeping the online students engaged in the tours and discussions during the 
live meetings. In some cases, the instructor was also the guide and led the discussions. At other 
times, the tours were led by local experts such as the live meeting at the Forum in Rome. For tours 
like this, the instructor started the meeting by situating the students' thinking and reviewing the 
learning objectives before turning the lesson over to the local expert. Throughout the tour of Rome, 
the online group was given the opportunity to interact by typing their questions for the guide 
through the Adobe Connect group chat. The online facilitator then asked these questions on their 
behalf as the students listened to the tour guide’s responses and gathered information. 

Study Population/Sampling 

The population for this study was 26 American students from a large U.S. university. The 
students were enrolled in two combined sections of the same intercultural communication course 
with a study abroad component to Italy. Ten students participated in the traditional study abroad 
trip while the other 16 participated online. Both sections received the same course content and 
were required to complete the same assignments. For the first three weeks of the 6-week intensive 
summer tenn, both sections met together in the same classroom and completed group activities 
with 2-3 students from each section per group. During weeks four and five, one section traveled 
to Italy (abroad group) while the other section joined the live meetings via technology (online 
group). During this time, one-hour live meetings were scheduled for each day. The final week of 
the course was online for both sections and was dedicated to completing the group project and 
final assignments. 

Data Collection 

To explore the evaluation questions and effectively evaluate a study abroad program, Rubin 
and Matthews (2013) recommended gathering data from multiple data sources. Replicating 
Howard and Gunter’s (2017) methodology, data sources included an anonymous student survey, 
instructor interview, session recordings, financial budgets, and online facilitator observations. For 
this study, additional researchers served as observers throughout the live meetings and reviewed 
the recordings when necessary. They also completed observation forms in order to record their 
experiences and capture what did and did not function well. This helped the researchers to properly 
identify technical challenges and perceptions of student engagement throughout the live meetings. 
Two additional sources were added: student interviews and Morais and Ogden’s (2010) pre/post 
global citizenship assessment. Collectively, these sources were used to triangulate and evaluate 
the effectiveness of this web-based intervention and extract effective practices and 
recommendations for improvement. 

Instrumentation. The anonymous online survey consisted of 45 Likert-scale questions 
with responses ranging from 1 (Strongly Disagree) to 5 (Strongly Agree) and was delivered to both 
groups of students at the end of the course. The online students were asked additional questions 
about the web conferencing technology and their online experience, and both groups were asked 
the same set of questions about their perceived learning and satisfaction. 
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The questions were adapted from multiple sources including the National Survey of 
Student Engagement (NSSE) developed by the Indiana University School of Education (2014) and 
Picciano’s (2002) survey instrument, which measured student satisfaction in an online 
environment. Also, researchers used the IBM Computer Usability Satisfaction PSSUQ 
Questionnaire for feedback on the technology (Lewis, 1993). In addition, Part I of Richardson and 
Swan’s (2003) and Picciano’s (2002) surveys were used to measure social presence in an online 
learning setting. The survey also contained 5 open-ended questions developed by the researchers 
to gather additional clarification. 

In addition to the anonymous online survey, the instructor was interviewed after the trip to 
record his perceptions, students were interviewed, financial documents were compiled to evaluate 
cost, researchers recorded observations after each live event to note student participation and 
acknowledge areas for improvement, and researchers reviewed the recorded sessions in order to 
triangulate the data in an effort to provide a comprehensive fonnative evaluation (Maxwell, 1996). 
Finally, the Global Citizenship instrument developed by Morais and Ogden (2010) was also used 
as a pre-assessment with 52 questions (including demographics) and a post assessment with 45 
questions to compare the two groups on ten factors organized under three dimensions: social 
responsibility, global competence, and global civic engagement. 

Data analysis. In addition to reviewing qualitative data from open-ended questions on the 
feedback survey, a t-test was run on the same two subscales established by Howard and Gunter 
(2017). The learning effectiveness (LEARNEFF) subscale ( ✓ = .92) was based on twelve items, 
and the satisfaction with experience (SATEXP) subscale (✓ = .86) was based on six survey items. 
The Global Citizenship instrument (Morais & Ogden, 2010) was also used to compare the changes 
in perceptions of the two groups. After running a t-test for significance, Cohen’s effect size was 
also calculated for practical significance. 

For all qualitative data sources (instructor interview, student interviews, and observation 
forms) one of four researchers transcribed the interviews and independently open-coded to reveal 
emerging themes, noting anything that related specifically to the evaluation questions. Then a 
second researcher independently verified the transcriptions and performed a second independent 
round noting themes specific to each evaluation question. Finally, the team reviewed and agreed 
upon the themes. 


Results 

Access 

Evaluation Question 1. What is the potential impact of this program for providing students 
access to international learning opportunities? The access to international learning 
opportunities for students seems to have been impacted in two ways: 1) use of technology, and 2) 
inspiration to travel abroad. Live meetings via technology provided access to an international 
experience for 16 online students who otherwise would not have been able to participate. It is 
important to note that the technology did not limit online enrollment to only 16 students — that 
was the number of students who chose to register for the online section of this class. Additional 
online enrollment was possible in tenns of the technology. 

The average online attendance for the seven live meetings was approximately 75% with 
11.5 students participating out of the 16 enrolled students. An online student shared the 
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significance of this experience, “All in all this class was a very demanding experience, but a class 
that has left an impact on me, and to me, that’s all you can really ask for in a course.” 

According to the feedback provided, the use of technology to allow access to international 
learning opportunities was successful. All students reporting via the feedback survey were satisfied 
at some level with the Adobe Connect software used for the live meetings. One hundred percent 
agreed that even though they were not physically together in a traditional classroom, they still felt 
like they were part of a group in the live meeting, and two-thirds of students agreed that the live 
meeting provided a personal experience similar to the classroom. Finally, the online students 
reported that their level of learning that took place in this meeting was of the highest quality. 

On the feedback survey, both online and abroad students agreed that this program helped them 
to: 

• Think critically and analytically: 83% online students and 88% abroad students. 

• Understand people of other backgrounds: 83% online students agreed and 100% of 
abroad students. 

• Be an informed and active citizen: 100% of online students and 75% of abroad students. 

These findings suggest that the use of technology in this case evaluation did increase student 
access to impactful international learning opportunities. 

Student Satisfaction 

Evaluation Question 2. What is the difference in student satisfaction with the overall 
experience between the online and abroad groups? The qualitative data revealed two main 
categories for feedback on student satisfaction: 1) the concept of connecting students who travel 
abroad with those participating online, and 2) the setting of the live meetings. The overall 
experience for students who traveled abroad and those who participated online were similar in 
their interaction with others during the program (instructor, experts in the field, online facilitator, 
other students). There were some differences between the groups for student perception of learning 
quality and learning expectations. 

The most recurring area of dissatisfaction for the abroad students was due to the setting 
(both location and time) of the final live meeting. It took place on the last evening of the trip during 
the group dinner. Originally the instructors planned for this to be a family-style dinner discussion. 
However, the restaurant was crowded and loud which made it impossible to do a traditional live 
meeting. Instead, the iPad was passed around the table so that group members abroad could directly 
communicate with their group members online. The abroad students did acknowledge that the live 
meeting was interactive with their online classmates; however, they also expressed frustration at 
not being able to focus on the food, wine, and people physically at the table in Italy. One of the 
students abroad commented, “The idea of this broadcast is great, however the location and timing 
was not. Students in Italy were hungry, wanted to eat and reflect with other students who travelled 
abroad.” 

The data provided by the students via the feedback survey showed different levels of 
satisfaction with some major aspects of the program: 

• My level of learning that took place in this meeting was of the highest quality. 100% of 
online students agreed with this statement while 62% of abroad students agreed. 

• Overall this session met my learning expectations. 100% of online students agreed with 
this statement and 62% of abroad students agreed. 
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Other data provided by the students via the feedback survey show similar levels of satisfaction 
with other major aspects of the program: 

• 100% of online students reported an excellent experience with the instructor and 100% of 
abroad students reported with a Very Good or Excellent experience. 

• 100% of online students reported a Very Good or Excellent experience with Experts in the 
Field and 75% of abroad students reported an Excellent experience. 

• 100% of both online and abroad students reported a Very Good or Excellent experience 
with the online facilitator. 

• 100% of both online and abroad students reported a Good to Very Good experience with 
other students. 

In addition, the satisfaction with experience (SATEXP) subscale (a = .86), which was based on 
six items in the feedback survey showed that there was no significant difference in satisfaction 
with the overall experience between the abroad group (M = 23.75, SD = 4.86) and the online group 
(M = 26.33, SD = 2.25) t(14) = .25, ns. Despite some differences in student satisfaction between 
the groups for learning quality and learning expectations, the overall experience of interacting with 
others during the program were similar for students who traveled abroad and those who 
participated online. 

Evaluation Question 3. What were online participants’ reactions to the web conferencing 

system? According to the researcher observations, the web conferencing system was not as 
interactive as they had hoped and they noted the online students had more discussion in the chat 
with the online facilitator speaking softly as compared with the students physically present in the 
room who were mostly silent out of respect for the tour guide. 

In the feedback survey, online students reported feeling comfortable using the Adobe 
Connect technology. They chose to communicate through the chat feature because it was easy to 
use and manageable with the online facilitator either repeating questions to the tour guide or 
immediately responding directly to the students. Researchers observed that at one point a student 
tested her microphone, but there was a lot of background noise which made it difficult to hear well. 

The data provided by the students via the feedback survey (Question 8) showed different 
reactions to the web conferencing system. 

• 100% of the online students agreed or strongly agreed that they felt comfortable 
conversing through this medium. 

• The students felt comfortable introducing themselves in the online environment and 
agreed that the instructor created a feeling of an online community. 

• The students overall felt that they were satisfied with the usability of the system because 
it was simple and effective to complete the tasks. 

They also commented that the system had all the functions and capabilities that met their 
expectations. However, there was one student out of a total of six respondents that said that 
learning to use the system was not easy. 

Some data showed that the live sessions met their learning expectations. 

• The students were neutral that the online instructional activity stimulated their desire to 
learn. 

• The live meetings allowed them to express their feelings and leam. 

• The students did not find the online meeting threatening. 
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• Quality of interactions with the instructor, experts in the field, the online facilitator, and 

the other students were indicated overall as good or excellent. 

One student responded that being able to see remarkable historical landmarks in real time 
surrounded by people they had previously met and were actively communicating with was really 
interesting and enjoyable. 

In one of the interviews, a student said that it was interactive and the experience was great 
being able to connect live, ask questions, and see with their own eyes. Another noted that the 
scenery from Italy was probably their favorite part of viewing the online meetings. Students liked 
the interaction between not only people of different groups, but different cultures. They thought it 
was a cool concept to engage a group from around the world and hear different perspectives and 
points of views. 

Student feedback indicated areas of improvement for the technical aspects of the web 
conferencing system. This feedback will be reviewed in Evaluation Question 9 and the Discussion 
sections of this article. 

Evaluation Question 4. What was the impact of technology on the students’ ability to 
participate in remote instructional activities? Technology does appear to have impacted the 
students’ ability to participate. The online facilitator observed that the online students were very 
participative and started conversations with one another in the chat which they preferred to use 
over their microphones. They asked great questions and wanted to hear responses not only from 
the instructor, but also from their classmates too. Therefore, technology enabled the online group 
to connect remotely with other students, the instructor and online facilitator, and experts in the 
field. 

Based on feedback survey Questions 11 and 12, online students reported that there were 
some limitations at times regarding the quality of the sound. There were times when the video 
streaming was not working smoothly which hindered the online group’s participation and noise of 
various types interfered with the technology. One student suggested, “The last meeting shouldn’t 
take place in an environment that is noisy. It was hard to interact this way. I would suggest next 
time hold it in a quiet setting.” Another student commented, “There were a couple of moments 
where the Wi-Fi wasn’t as great and so it wouldn’t show a good connection or picture. Overall 
though the instructors were really great at getting the connection back and explaining what we 
missed.” Online students also experienced a longer delay compared to the abroad group when 
responding to the instructor’s questions: they had to first consider their response, type it, and then 
wait for the online facilitator to relay the message. 

According to feedback survey Question 6, only one online student replied that they felt 
uncomfortable conversing through this medium. Overall, all the online students reported that they 
felt comfortable introducing themselves in the online environment and the instructor performed 
well with creating a feeling of an online community. One online student disagreed that an online 
meeting allowed for social interaction or provided them a reliable means of communication, 
however, overall, the online students replied in the survey that their experience in a live interactive 
session was enjoyable and it helped them stimulate their desire to leam. 

While several online students expressed a desire to have joined the others abroad, one 
online student noted a benefit of technology, “I paid considerably less and was able to continue to 
take other classes...” They also felt emotions transferred through the live screen as if they were in 
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Italy. According to the student surveys, some online students said that the technology affected their 
learning because they could see and experience the difference between cultures halfway around 
the world. 

One challenge of using technology to connect students in live meetings across cultures and 
time zones is the need to plan and coordinate schedules. One online student noted the importance 
of “knowing the schedule of them being abroad ahead of time, I had to really fix my schedule 
around it.” Also, in the feedback survey Question 18, one student abroad mentioned that at times 
it was hard to get organized regarding assignments with the online group. 

Faculty Satisfaction 

Evaluation Question 5. What were the instructor’s perceptions about his teaching experience 
with this technology-mediated approach? The instructor reported the overall experience with 
the program as being positive. However, there were some areas of dissatisfaction. After reflecting 
upon the physical and online technicalities of the program, the instructor explained areas of 
concern and provided additional suggestions for improvement of future programs. 

Pertaining to curriculum development, the instructor offered these suggestions for future 
improvement. He was dissatisfied with some of the outcomes of the group activities and would 
suggest that discussion prompts for students be prepared prior to the beginning of the study abroad 
experience. He felt that this would increase dialogue and facilitate connection between the abroad 
and online groups. The instructor suggested that this type of curriculum be designed ahead of time 
so that it is reinforced during group work to improve the learning experience of both the 
participants abroad and online. When in Italy with the students, the instructor would have prepared 
scripts for the live meetings and precisely mapped out the route of the abroad group prior to travel 
in order to create a more organized and less stressful teaching experience. 

The instructor found it challenging to engage both the abroad and online groups 
simultaneously during live meetings. He explained, “Yeah, it was a real challenge for me because 
I found myself first catering a little bit more to the online group, like talking to the camera. And 
then some of the feedback from some of the in-country students noted that I was paying more 
attention to the camera. And then I tried to flip it, and I felt that I was not giving - I felt 
disconnected from the online students. For me, it was a real challenge trying to divide my attention 
between the two groups.” 

Finally, the instructor expressed complete satisfaction with his online facilitator throughout 
the program and process of this research study. He was particularly pleased with her organization, 
technical skills, and overall contributions to the success of the online abroad program and its 
participants. He was also satisfied with the university’s study abroad office. The instructor 
expressed that the study abroad office handled the logistics of the travel portion of the course well 
and took his ideas and objectives for the course and made it a reality. 

Learning Effectiveness 

Evaluation Question 6. What is the difference in student engagement with activities and 
experts in the field between the abroad group and the online group? Five categories of student 
engagement emerged from the qualitative feedback. The feedback for 1) quantity of questions, and 
2) interaction with the technology, suggests that there were some differences in student 
engagement between the groups. These differences are explained in this section. Note that the 
feedback for the other three categories 3) quality of assignments, 4) communication between the 
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abroad and online groups, and 5) interaction with the instructor and experts in the field, indicates 
that there was not a noticeable difference. 

Quantity of questions = differences in student engagement. According to the instructor 
interview, online students asked more questions than those abroad during the live meetings. In the 
student interviews, one of the abroad students said that they appreciated when the online students 
asked questions as it stimulated conversation and brought in fresh perspectives. Abroad students 
also liked when those online asked them questions instead of directing everything to the guide, 
instructor, or online facilitator. 

Interaction with the technology = differences in student engagement. The technology 
appears to have created some perception of difference in engagement. A researcher noted in their 
observations that the abroad students who were randomly put ‘on the spot’ with the iPad in their 
direction seemed intimidated and would not talk as much as when there was no pressure with the 
camera. In the feedback survey, a student mentioned that they felt it hindered both the abroad and 
online groups because the technology detracted from the natural communication setting. It is 
important to note that some students commented that the technology was neither a help nor a 
hindrance and “was just kind of there.” 

However, one hundred percent of the online students who participated in the feedback 
survey, agreed with the statement, “I enjoyed the online instructional activities.” The online 
students reporting via the feedback survey said that the Adobe Connect web conferencing system 
was a beneficial tool in three main ways: 

• Video stream from Italy was Very Important or Critical 

• Audio stream from Italy was Critical 

• Text chat was Very Important or Critical 

The feedback survey responses from both the online and abroad students indicate that the 
majority (100% online and 87% abroad) of both groups felt comfortable participating in the group 
discussion and felt comfortable interacting with other participants in the live meeting. 

Evaluation Question 7. How were the learning experiences of the online and abroad groups 
similar and how were they different? While the students who traveled to Italy had a far more 
immersive experience, both groups demonstrated learning gains. The global citizenship pre/post 
assessment measures seven subscales: social responsibility, self-awareness, intercultural 
communication, global knowledge, involvement in civic organizations, and political voice (Morais 
& Ogden, 2010). Table 1 displays the subscales where each group demonstrated significant 
improvement including practical significance based on Cohen’s effect size. Both groups showed 
improvement in self-awareness and intercultural communication, but the students who traveled to 
Italy also showed improvement in global civic activism and global knowledge. 
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The abroad group demonstrated significant improvement on the following scales: 


Cohen’s effect size 

Practical significance 

Global Civic Activism 

d = .54 

Moderate 

Self-awareness 

in 

<n 

II 

"d 

Moderate 

Intercultural Communication 

d = .63 

Moderate to High 

Global Knowledge 

d= 1.48 

Very High 

The online group demonstrated significant improvement on the following scales: 

Self-awareness 

d = .63 

Moderate to High 

Intercultural Communication 

<N 

in 

II 

"d 

Moderate 


Table 1. Significant Improvements on Global Citizenship Pre/Post Assessment 


When comparing the two groups, the online students appear to have improved more on the 
social responsibility scale. There was a statistically significant difference between the two student 
groups, students abroad (M = -1.22, SD = 2.10) and students online (M = .267, SD = 1.27), t(l, 
22) = 4.69, p < .05. Further, Cohen’s effect size value (d = .86) suggested a large practical 
significance. Conversely, the students abroad appear to have improved more on the global 
knowledge scale. There was a statistically significant difference between the two student groups, 
students abroad (M = 2.11, SD = 2.26) and students online (M = .40, SD = 1.81), t(l, 22) = 4.19, 
p < .05. Further, Cohen’s effect size value (d = .84) suggested a large practical significance. There 
was no statistically significant difference between the two groups on the remaining five scales 
(self-awareness, intercultural communication, global knowledge, involvement in civic 
organizations, or political voice). 

On the feedback survey, the learning effectiveness (LEARNEFF) subscale (a = .92) also 
showed no significant difference between the students who participated face-to-face in Italy (M = 
52.88, SD = 7.26) and those who participated online (M = 52.67, SD = 5.09), t( 14) = .95, ns. This 
subscale consisted of 12 items that asked about the quality of their interactions with the instructor, 
experts in the field, the online facilitator, and other students in addition to the impact on their 
knowledge, skills, and personal development in the following areas: 

• Speaking clearly and effectively 

• Thinking critically and analytically 

• Acquiring job- or work-related knowledge and skills 

• Working effectively with others 

• Developing or clarifying a personal code of values and ethics 

• Understanding people of other backgrounds (economic, racial/ethnic, political, religious, 
nationality, etc.) 

• Solving complex real-world problems 

• Being an informed and active citizen 
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During the instructor interview, he elaborated on how the learning experiences between the 
two groups were similar and different. For similarity, he felt that the, “experiences were the same 
in that the curriculum was the same. They both had to explore the projects and the theories from 
the same basic standpoint..Also, the instructor observed that the “in-field experts, the way they 
interacted with the face-to-face students and the online students were very similar in my opinion. 
I didn’t notice a great difference.” 

The instructor did think there were some differences in learning experiences. He 
explained: 

So how I think they were different: the students who were in country got to use all five of 
their senses where of course the ones at home just got two — sight and sound. And then 
from the student feedback based on culture shock, it gave some insight on how the 
experience could have been different as well. The students who were abroad talked about 
how changed they were... And the online students, they either had to stretch to answer that 
question and refer back to experiences they had from different travels on their own or they 
just said, I didn’t travel abroad so I didn’t experience culture shock. 

Despite the similarities and differences of the learning experiences of the online and abroad groups, 
both provided evidence of learning gains. 

Scale 

Evaluation Question 8. How does the cost of this case compare to previous technology- 
mediated attempts? Every effort was made to minimize cost in order to develop a scalable, 
sustainable approach. The initial proposal for the pilot study made use of satellite technology, 
which also required a crew to operate. Because that budget was cost prohibitive, a new Wi-Fi based 
approach with Adobe Connect web conferencing software and one online facilitator was actually 
implemented. Table 2 compares the original proposed costs, the pilot study cost, and the actual 
cost of this study. 



Satellite Proposal 

Brazil Pilot: Actual 

Italy Study: Actual 

Travel 

expenses: 

• 4 webcasts 

• 10 days/9 nights 

• 1 faculty member 
+ 3 grad, assistants 

• 5 webcasts 

• 10 days/9 nights 

• 1 faculty member 

• Use existing University 
resources for hardware & 
software 

• 8 webcasts 

• 14 days/13 nights 

• 1 faculty member 

• Use existing University 
resources for hardware & 
software 

Cost to 
broadcast: 

$8,085 

$3,400 

$5,699 

Hardware: 

$19,000 

$180 

$245 

Software: 

$14,700 

$0 

$0 

TOTAL: 

$41,785 

$3,580 

$5,944 


Table 2. Cost Comparison 
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Each course will carry a variable cost based on factors such as the duration or location, but 
it closely approximates the cost for an individual student enrolled in the travel option. For these 
studies, costs were reduced by utilizing an existing Adobe Connect license and equipment such as 
iPad, laptop, and webcam. 

Evaluation Question 9. How can this approach be improved? 

Critical Feedback. In addition to positive feedback, the online survey and interviews 
provided clear areas for improvement. Some were more pedagogical in nature while others were 
technical. The technical areas of concern included audio and Internet connection quality. In the 
feedback survey, one student suggested, “have the tour guide wear a microphone of sorts just 
because it was really hard to hear at times.” Another responded with “The last meeting shouldn’t 
take place in an environment that is noisy. It was hard to interact this way. I would suggest next 
time hold it in a quiet setting.” Three observers also noted concerns about background noise. For 
example, one reported “since the broadcast took place in a restaurant, noise disruptions were 
present for online students” and another recorded “this broadcast would have worked perfectly if 
it was done in a quiet setting without many distractions.” The online facilitator also noted in an 
observation form “I had to stay within a few feet of the tour guide to make sure online students 
could hear well. Possibly consider a better microphone in the future.” 

During two of the live sessions the Wi-Fi disconnected briefly. While students were 
prepared for this possibility in advance, it is never desirable. In an interview one student explained, 
“Every now and then the live sessions would pause or cut out and I know that is just the way 
technology works but maybe with technological advances it would be possible to have a better 
connection in the future.” Whenever possible, the instructor and online facilitator visited the sites 
in advance to test the connection and prepare contingency plans. 

One of the pedagogical concerns was related to efficient use of time and engaging students 
in constructive dialogue during the live sessions. The online facilitator observed that during the 
tours often there were stretches of time where the group in the field was walking from one stop to 
the next, and during that time she felt the need to improvise and fill that time with group discussion 
to keep the online students engaged and encourage student to student interaction. Student survey 
comments such as “have more interactive questions” and “It would have been nice if there was a 
little more participation from the students in Italy” justify the need for preparing discussion 
prompts in advance. One of the researchers suggested, “As an icebreaker, encourage dialogue 
between online students with those abroad regarding initial thoughts about the trip. This would 
also probably make students abroad feel more comfortable talking in front of a camera.” The 
instructor also reported challenges trying to balance the two different audiences and give them 
equal attention, and prepared discussion prompts may be directed to the entire group for a more 
inclusive experience. 

The other pedagogical concern focused more on group dynamics. In this particular course, 
two students in the field were paired with three online students to complete a cultural comparative 
group project. In their interviews, several students who traveled abroad reported that they 
experienced challenges working with their online partners at a distance and would have preferred 
to work just with their fellow travelers with whom they had bonded. For example, one student 
stated, “I feel like it would’ve been easier for all of us in Italy to have done a project together and 
then everyone in Florida to have done a project together so we could talk face-to-face about our 
project.” While this would negate one of the primary learning objectives of the course (using 
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technology to collaborate at a distance), it does indicate that the group fonnation process was 
rushed prior to physically separating. In their interviews, several students requested more group 
time in class prior to the trip. 

Recommendations for improvement. The feedback provided by both sets of students, the 
instructor, the online facilitator, and observers resulted in the following recommendations for 
improvement: 

• Improve audio quality. Often background noise in the field made it difficult to hear the 
instructor or tour guide clearly. While some noisy environments like restaurants may add 
to the immersive cultural experience, it should not be allowed to negatively impact 
instruction. This could be improved by using a directed microphone or a lavalier 
microphone on the instructor or meeting in a private room rather than in the main dining 
room of a restaurant. 

• Test Wi-Fi strength in advance. It may not always be possible to test in advance and 
sometimes the signal suffers during the live event even when the test went fine, but every 
effort should be made to minimize this risk. Testing signal strength in the various locations 
also gives the instructor and online facilitator an opportunity to work out backup plans, but 
they should also be prepared to improvise when unforeseen connection issues arise. 

• Prepare question prompts. Often there are brief downtimes in the field when the group 
is moving from one location to another. Together with the instructor, the online facilitator 
could prepare discussion prompts in advance in order to make effective use of that time 
and promote student engagement. 

• Establish team roles, communication protocols, and dynamics. Provide groups more 
time to connect and bond during the weeks prior to splitting up and assist with the group 
fonnation process. This may include requiring pre-travel group activities or assignments 
in class, providing roles for individual group members, and coaching or structured 
guidance as they work in their teams. 


Discussion 

Explanation of the case evaluation 

The 6-week course “Communication, Culture and Technology: An Italian Case Study” 
combined a traditional study abroad experience with an innovative twist: live meetings for students 
who have chosen to join the group abroad virtually. The overall goal was to make study abroad 
more accessible to students in order to promote the acquisition of global competencies. Building 
upon the work of Howard and Gunter (2017), the introduction of technology brought the abroad 
experience into the homes of students who would otherwise miss out on a valuable international 
learning opportunity. 

Two student groups were enrolled in the course: an abroad and online group. Both groups 
met together on campus for the first three weeks prior to traveling abroad. When designing the 
curriculum, both student groups needed to have the same course objectives. To accomplish this, 
the assignment activities of the course required that classmates work in groups composed of both 
abroad and online students to produce a module on some aspect of culture and communication. 
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Findings from the case evaluation 

Students liked that they were able to hear different perspectives and points of views. They 
thought it was a cool concept to engage with a group around the world to have them experience 
Italy from their professor's perspective and ask questions to the class. 

Students reported the challenge of feeling emotionally connected to their group members 
who were not physically with them. The abroad students naturally bonded with one another 
through shared emotional experiences from their Italian travels. While the abroad students did 
collaborate with their online group members on the projects, the nature of the relationship seems 
to have been more information-focused than relationally-focused which does not naturally lead to 
bonding through shared emotional experiences. 

A possible hindrance between the groups was that the online students liked to use chat and 
chose not to utilize a microphone during the broadcasts. Therefore, the online facilitator had to act 
as a liaison and connect the two groups by repeating the online student questions aloud to the 
abroad group. They were not directly talking to one another. 

The online students asked more questions than those abroad during the live meetings. An 
explanation of this could be that the abroad students were with the instructor 24 hours a day and 
had access to ask questions at any time and not be limited to just the live meetings. Online students 
would either need to email the instructor with a question or ask during the live meetings. 

Finally, the instructor found it challenging to engage both the abroad and online students 
simultaneously during live meetings. Stronger relationships seem to have been fonned with the 
abroad students than the online students. The instructor suggests that the relationship-building 
aspect that comes from the travel abroad experience may not be able to be replicated via 
technology. 

Recommendations for future case evaluations 

While this case evaluation included just one online section, it could be possible to increase 
the number of online sections to allow other related courses to join the live meetings for specific 
tours that are related to their coursework. When replicating similar programs in the future, this 
case evaluation generated several recommendations: 1) improve audio quality 2) test Wi-Fi 
strength in advance 3) prepare question prompts, and 4) establish team roles, communication 
protocols, and dynamics. 

A Bluetooth microphone might improve the audio quality during the live meetings. Wi-Fi 
should be tested prior to broadcasts to detennine its strength both indoors and outdoors. The 
instructor can plan alternative lessons in the event that the Wi-Fi does not permit broadcasting. 
The lesson should include question prompts to engage both abroad and online students during the 
live meetings. Finally, official roles for each group member could help the teams establish better 
dynamics. More relationship-building exchanges could help create a stronger bond between abroad 
and online group members as well as with the instructor. Also, allowing the student groups to 
interact directly via the iPad can allow the abroad students an opportunity to experience the role 
of the online facilitator as an additional benefit. 
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Conclusion 

The purpose of this study was to evaluate the effectiveness of an online delivery model for 
study abroad activities. Twenty-six undergraduate students studying intercultural communication 
were able to participate in a study abroad experience in Italy: 10 students participated in the 
traditional study abroad trip while the other 16 participated online. This case evaluation continues 
the cyclical, design based research started by Howard and Gunter (2017) to refine and further 
develop the web conferencing approach to internationalizing the curriculum through online 
instruction. 

The study was driven by three guiding questions: (1) Did the program accomplish its goals? 
(2) What was the impact of how it was implemented? (3) What changes to the design are necessary 
to firmly establish it as a valuable alternative to traditional study abroad? These questions were 
evaluated through a combination of anonymous surveys, pre/post assessments, observations, and 
student and instructor interviews. 

Overall, the program accomplished its goals and provided a valuable international learning 
opportunity. While the students who traveled to Italy had a more immersive experience, both 
groups demonstrated learning gains. Both groups showed improvement in self-awareness and 
intercultural communication. These findings expand the established history of study abroad 
programs by furthering demonstrating the benefits for students (Chieffo & Griffiths, 2004; 
McKeown, 2009; Spencer & Tuma, 2002; Tarrant, Rubin, & Stoner, 2014; Vandeveer & Menefee, 
2006). 

The way the study was implemented did have an impact. The students who traveled to Italy 
showed improvement in global civic activism and global knowledge. The online students were 
able to join the live tours and discussions, thus expanding their access to international experiences 
that normally would be closed to them. One online student thought that the live meetings were, “a 
good enough way to experience another culture while not having to spend the extra money.” 
Another student thought that the university “should offer more classes like this for people who 
don’t have the opportunity to go abroad. It’s a good opportunity to get the experience without 
being there.” Other students referred to the live meetings as a way to, “explore Italy side by side 
with those who went abroad.” 

The program inspired students from both the abroad and online groups to travel and explore 
cultures. One student said, “This class has made my desire to travel stronger and I feel I am better 
prepared than before.” And another commented, “I would still do it again. I would still want to do 
the long dining restaurants...” 

Finally, the research revealed some changes to the design that are necessary to firmly 
establish the online option as a valuable alternative to traditional study abroad. Future programs 
implementing a similar model that connects online and abroad students should 1) improve audio 
quality 2) test Wi-Fi strength in advance 3) prepare question prompts, and 4) establish team roles, 
communication protocols, and dynamics. 

Based on these findings, the instructor and online facilitator for this case evaluation 
encourage future program leaders or facilitators with a passion for study abroad to consider 
implementing an online model to increase student accessibility for international learning 
experiences. 
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